ESQL: Document LU JOIN/MV_EXPAND not respecting SORT #127718
Conversation
alex-spies
added
>docs
elasticsearchmachine
added
Team:Docs
|
Pinging @elastic/es-docs (Team:Docs) |
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
| :::: | ||
|
|
||
| ::::{warning} | ||
| The output rows produced by `LOOKUP JOIN` can be in any order and may not |
There was a problem hiding this comment.
@leemthompo , should we also place this warning in the more in-depth landing page for lookup join?
There was a problem hiding this comment.
I think we are running the risk of those two pages becoming undifferentiated and therefore hard to maintain, if we repeat everything in both. Also adding admonitions like this doesn't scale.
We could consider moving this warning and the tip above it into the landing page under a new heading like Usage notes. And then we can link to that new section of the landing page from here with a message like "Refer to Usage notes for important information about using the LOOKUP JOIN command"— instead of repeating admonitions on both pages. This will be cleaner, more future-proof, and easier to maintain.
WDYT?
There was a problem hiding this comment.
I like the new structure. Thanks, Liam!
|
|
||
| ### Handling name collisions | ||
|
|
||
| In case of name collisions, the newly created columns will override existing columns. |
There was a problem hiding this comment.
@alex-spies @costin this is pre-existing info unchanged in this PR, but was wondering if there was additional guidance about workarounds we could give for dealing with name collisions?
There was a problem hiding this comment.
Sure, good idea!
Like this?
Such columns can be preserved by using `RENAME` to assign non-conflicting names before the `LOOKUP JOIN`.
And yes, we realize this is clunky and we want to have a more ergonomic approach to avoid this renaming dance.
|
|
||
| For a complete list of supported data types and their internal representations, see the [Supported Field Types documentation](/reference/query-languages/esql/limitations.md#_supported_types). | ||
|
|
||
| ## Implementation details |
There was a problem hiding this comment.
Hm, that's not really an implementation detail, that's more the semantics that the command provides.
Maybe "behavior details" or "semantic details" or "results" or something along these lines?
There was a problem hiding this comment.
Yup agreed that was rough placeholder— we'll go with Usage notes for future-proofery :)
Co-authored-by: Alexander Spies <[email protected]>
leemthompo
added
the
auto-backport
(cherry picked from commit 9e3ae5b)
(cherry picked from commit 9e3ae5b) Co-authored-by: Alexander Spies <[email protected]>
3 participants
Relates #121884.
We currently cannot run all queries that have a
SORTbefore eitherLOOKUP JOINorMV_EXPAND. This is because we cannot push down these commands pastSORTs without breaking the sort order in some cases. We may or may not want to choose to push down in these cases in the future, even though this affects sort order.Moreover, further work and generalizations of
LOOKUP JOINmay be even less compatible with sorting - e.g. if we generalize lookup indices to have multiple shards residing on different nodes.To keep all possibilities open, let's just document that we give no guarantees whatsoever; we can revisit this later when needed.